Alex Monahan writes that the open-weight Qwen 3.8 27B model, running locally via LM Studio on a consumer laptop, achieves frontier-level agentic SQL performance at essentially zero marginal cost. On the DABstep benchmark (400+ questions), the locally-run 4-bit quantized model outperformed OpenAI's GPT 5.6 Luna Max at 17x lower cost, and a 3-bit quant still worked on a nearly five-year-old M1 Pro MacBook Pro with only 16GB RAM. The setup pairs the local LLM with DuckDB for query execution, with MotherDuck's cloud hypertenancy as an optional escape hatch for scale.
- MTP (Multi-Token Prediction) yields ~30% throughput boost on M5 hardware but actually slows down older M1 Pro chips
- Runtime remains the main gap: 5–6 min per question locally vs. 25–40 sec for cloud frontier models
- The benchmark context layer was built using a frontier model (Claude Fable 5); only the eval loop runs locally
- Roughly 1 in 30 Macs in the wild have the 16GB+ unified memory needed; most laptops don't qualify
- Including laptop depreciation, the cost rises to ~$6 per 1,000 questions answered
- MTP (Multi-Token Prediction) yields ~30% throughput boost on M5 hardware but actually slows down older M1 Pro chips
- Runtime remains the main gap: 5–6 min per question locally vs. 25–40 sec for cloud frontier models
- The benchmark context layer was built using a frontier model (Claude Fable 5); only the eval loop runs locally
- Roughly 1 in 30 Macs in the wild have the 16GB+ unified memory needed; most laptops don't qualify
- Including laptop depreciation, the cost rises to ~$6 per 1,000 questions answered
